Papers with deficient interaction
Pre-training Language Model as a Multi-perspective Course Learner (2023.findings-acl)
Copied to clipboard
Beiduo Chen, Shaohan Huang, Zihan Zhang, Wu Guo, Zhenhua Ling, Haizhen Huang, Furu Wei, Weiwei Deng, Qi Zhang
| Challenge: | Experimental results show that our method significantly improves ELECTRA’s average performance by 2.8% and 3.2% absolute points respectively on GLUE and SQuAD 2.0 benchmarks. |
| Approach: | They propose a multi-perspective course learning method to fetch many degrees and visual angles for sample-efficient pre-training and to fully leverage the relationship between generator and discriminator. |
| Outcome: | The proposed method improves ELECTRA's performance on GLUE and SQuAD 2.0 benchmarks and overshadows recent advanced ELECL-style models under the same settings. |